† Corresponding author. E-mail:
Project supported by the National Natural Science Foundation of China (Grant Nos. 61471377, 61804181, 61604177, and 61704191).
Vector neural network (VNN) is one of the most important methods to process interval data. However, the VNN, which contains a great number of multiply-accumulate (MAC) operations, often adopts pure numerical calculation method, and thus is difficult to be miniaturized for the embedded applications. In this paper, we propose a memristor based vector-type backpropagation (MVTBP) architecture which utilizes memristive arrays to accelerate the MAC operations of interval data. Owing to the unique brain-like synaptic characteristics of memristive devices, e.g., small size, low power consumption, and high integration density, the proposed architecture can be implemented with low area and power consumption cost and easily applied to embedded systems. The simulation results indicate that the proposed architecture has better identification performance and noise tolerance. When the device precision is 6 bits and the error deviation level (EDL) is 20%, the proposed architecture can achieve an identification rate, which is about 92% higher than that for interval-value testing sample and 81% higher than that for scalar-value testing sample.
Interval data processing, an important technology, possesses broad application prospects in signal classification, speech recognition, image processing, etc. In many cases, the data to be processed are uncertain, and this uncertainty could be described by an interval or a vector. For example, for an electronic support measure (ESM) system, due to the diversity of the radar emitter parameters, the complexity of electromagnetic environment, and the error of measurement device itself, the measured radar emitter parameters, such as radio frequency (RF), pulse width (PW), pulse repetition interval (PRI), etc., would be expressed in the form of intervals.[1–3] Various approaches, including interval fuzzy logic system,[4] neural network,[5] are proposed to solve the classification problem of interval data. Among those methods, the vector neural network (VNN) is the most effective.[1,2] To the best of our knowledge, the current VNN implemented by software algorithm is mainly based on the general-purpose computing platform, which is a purely numerical calculation method without considering the problem of hardware implementation. As a result, the computation cost makes it difficult to realize the miniaturization and embedded system.
Memristive device possesses advantages of small size, low power consumption, and high integration density,[6–10] which are very suitable for building a brain-inspired computing system.[11–14] Therefore, various network architectures based on memristors, such as SNN, MLP, and CNN, have been proposed and achieved good results in many application fields.[15–19] However, the above networks can only handle scalar type data. When the input data is interval-value, those methods will not be workable.
To solve the above-mentioned problems of interval data classification, in this paper we propose a memristor-based vector-type backpropagation (MVTBP) architecture. Through integrating together the advantages of memristor and vector neural network, the multiplication and accumulation of interval type data can speed up effectively.
The rest of this paper is organized as follows. In Section
The MVTBP architecture is shown in Fig.
Figure
It should be noted that the input vectors V1 and V2 in Fig.
The above hardware architecture mainly realizes the multiplicative and accumulative operations of interval data. Taking the hidden layer of the 3-layer MVTBP network for example, and the neuron output of the hidden layer can be computed as follows:
![]() |
![](cpb_29_2_028502/cpb_29_2_028502_ieqn10.gif)
![](cpb_29_2_028502/cpb_29_2_028502_ieqn11.gif)
![]() |
![]() |
![](cpb_29_2_028502/cpb_29_2_028502_ieqn12.gif)
The main purpose of backward propagation of the MVTBP architecture is also to compute the weight (such as
In this section, the five-emitter-identification (EID) problem in Ref. [1] is taken for example to verify the performance of the proposed MVTBP architecture. For the five-EID problem, there are 50 interval-value and scalar-value training samples in the training step (each emitter type has 10 training samples) respectively, and there are also 150 interval-value and scalar-value test samples in the testing step (each emitter type has 30 testing samples). It should be noted that all the simulated emitter samples are from the radar manual[2] or the emitter samples repository,[1] which stores the measured emitters of five types. The parameters of the emitter samples are RF, PRI and PW, and can be seen in detail in Ref. [1]. So, these samples can represent the actual application situation to a certain extent.
In order to facilitate the hardware implementation, we normalize the emitters’ parameters, making them situated in a range between 0 and 1. In addition, to ensure the convergence of the network, we also increase the node number of hidden layers and employ the MVTBP architecture with a 3–30–5 network. For verifying the adaptability of the proposed MVTBP architecture to different measurement errors, it is also necessary to test interval emitter samples with additive noise at different levels. By introducing the error deviation level (EDL) proposed in Ref. [2], we can generate noisy testing samples by adding noise to the samples without noise. In this section, the noisy interval-value testing samples with different EDLs (from 0 to 20%) are given to verify the performance of the MVTBP architecture.
In order to verify the identification performance under a similar neural network architecture, the NVTBP algorithm is selected[2] as the comparison objects in this subsection. The identification performance on interval-value and scalar-value testing samples are shown in Figs.
Compared with the NVTBP algorithm, the MVTBP architecture has general advantages in both low EDL and high EDL situation as can be seen in Fig.
From Fig.
From Figs.
In terms of resource consumption, the MVTBP architecture requires only 480 (3 × 30 × 2 + 30 × 5 × 2) memristive devices, which is far less than those required by the traditional numerical calculation method, so the proposed architecture is very suitable for miniaturization and embedded applications.
Since the resistance state of the actual memristive device is limited, it is of great practical significance to discuss the radar emitter identification performances of the memristors with different precisions (i.e., the resistance state of the memristive device). Assuming that the adjustable precision of the memristive device is N bits, the adjustable state S of its resistance value is S = 2N. For example, when the precision N of the memristive device is 6 bits, the corresponding adjustable resistance state number is 26 = 64.
In this subsection, the identification performance of different N bits (i.e., Software, 10 bits, 8 bits, 6 bits, 5 bits, 4 bits) is discussed. The item Software denotes adopting default arithmetic precision of Matlab, that is, the double-precision 16-bit effective number. The identification results on interval-value and scalar-value testing samples are discussed as follows. As can be seen from Table
![]() | Table 1. Performances on interval-value testing samples with different precisions of memristive device. . |
As the precision N of the memristive device decreases, the identification rate of the MVTBP architecture does not change obviously at the beginning, and then suddenly drops at the ending. When N = 6 bits, that is, the number of adjustable resistance state is 64, it shows better identification performance, almost the same as the high precision case. When N = 5 bits, for interval-value testing samples, the identification rate is about 84%, and remains basically unchanged with the increase of EDL, while for scalar-value testing samples, the identification rate decreases with EDL increasing. When N arrives at 4 bits, the identification rate drops to about 50%–60%. It can be seen from the above analysis that in order to make the emitter identification have better performance, the condition that the precision N of the memristive device should be at least 6 bits is required to be satisfied.
![]() | Table 2. Performances on scalar-value testing samples with different precisions of memristive device. . |
For the feasibility of the MVTBP architecture, we verify the identification performance and analyze the influence of different device precision. Besides, we have designed neuron circuits to generate the impulses needed for the network and built a quasi-analytical model of three-dimensional (3D) vertical-RRAM array architecture for MB-level design in the early stage,[20] in which discussed were the influences of array device parameters, such as resistance state, device non-linearity, reading voltage, array scale, etc., on reading and writing performance. All of these can provide a benchmark for the practical application of this architecture. In addition, with the development of device fabrication technology, the performance of memristive array can be improved. Therefore, we believe that the proposed MVTBP architecture will be feasible in practice in the future.
In summary, this proposed MVTBP architecture, can successfully process the interval-value and scalar-value data with noise, and is very suitable for building a specialized embedded system, such as electronic reconnaissance system, speech recognition system, image processing system, etc., because in many cases, these systems have relatively high requirements for size, power consumption, etc., and the measured input data are generally uncertain.
Aiming at the identification problem of interval type data and the requirements of embedded application, in this paper we design an MVTBP architecture, which can realize multiplicative and accumulative operations of interval type data. The results of simulation experiments verify the performance of the proposed MVTBP architecture, and show that the proposed architecture has better identification performance and noise tolerance, and the device precision of 6 bits can meet the requirement of emitter identification. The research in this paper can also provide support for extending memristor-based network applications.
Of course, there is also some room for further improvement of the MVTBP architecture, such as the resistance fluctuation, yield, etc. In addition, considering other cases such as image classification, the designing of different network architectures is also an important research hotspot. These problems can be realized by improving the device preparation process, optimizing the network architecture design and the like.
[1] | |
[2] | |
[3] | |
[4] | |
[5] | |
[6] | |
[7] | |
[8] | |
[9] | |
[10] | |
[11] | |
[12] | |
[13] | |
[14] | |
[15] | |
[16] | |
[17] | |
[18] | |
[19] | |
[20] |